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I. introduction 



Estimating the scale and scope of training has be- 
come something of a growth industry. As part of the 
national debate over the^requirements for economic 
revitalization, there has been a marked increase in the 
importance accorded to job training, youth apprentice- 
ships, and schools that prepare their students for the 
labor market. In order to better understand the nation's 
current investments in work-related education and 
training, this interest has been accompanied by a sub- 
stantial increase in the number of surveys that have 
asked workers, both present and future, how often and 
from whon: have they received work-related training. 

In one respect, however, trainings new importance 
has proved to be too much of a good thing. As the ef- 
fort to measure training and its effects has gathered 
momentum, an increasingly sophisticated array of na- 
tional surveys yielded results that were not just widely, 
but wildly, different in their estimates of who received 
training and why. The more diligently the research 
community worked to establish a baseline for training 
rates, the less certain what qualified as work-related 



education and training became — much less the mea- 
surement, even the approximation, of its incidence. 
How, for example, was one to make sense of the fact 
that two well-constructed surveys with almost identical 
sampling frames and administered less than a year 
apart reported that the proportion of the working popu- 
* lation who had received training was either two-thirds 
or one-quarter— take your pick? What was the nation 
being told when more than 70 percent of the respon- 
dents to a survey designed by the Bureau of Labor Sta- 
tistics reported that they needed no training— including 
on-the-job training — to perform their current jobs? 

Our purpose in this essay is to attempt to explain 
why these estimates of training have proved so variable 
and, in that sense, unreliable. We have proceeded in 
this effort, much as the historian or archeologist might, 
by treating national surveys and the responses they 
elicited as artifacts generated by a process in which we 
believe training appeared to be as much a mirage as a 
reality. What we sought is an understanding of how 
these data came to be — what processes and transac- 
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lions gave shape to the responses at hand. From this 
vantage point we could then ask, "What do these con- 
flicting training estimates tell us about the perception 
and meaning of training in the American context?" To 
answer our question, we needed a better understanding 
of what the respondents thought they were being asked 
> arid why, in the end, they responded so uncertainly. 

The surveys we examined — the archeological "tells" 
of our excavations — included major national surveys 
sponsored by the federal government since 1973. Each 
survey was based on a national probability sample in 
which either individuals or individual households were 
asked, among other questions, to report their income, 
age, gender, ethnicity, education, and whether they had 
received work-related education or training. Many of 
the surveys were part of the regular Current Population 
Survey (CPS) series — in this case, the May Triennial 
Adult Education Supplement that asked whether mem- 
bers of the household had participated in a broadly 
defined range of adult education programs that includ- 
ed work-related education or training, as well as two 
special CPS surveys that included supplements dealing 
with work-related training. Three of the surveys were 
longitudinal analyses that followed specific youth co- 
horts — two were developed by the Department of Edu- 
cation and the other by the Bureau of Labor Statistics. 
Some of the surveys had a special focus, cither training 
itself or participation in a range of federal programs 
that included training as well as social security and 
public welfare. In the latter case, the survey sought to 
measure the incidence of training whether or not it was 
part of a federal program. 

In the course of our research we became the first to 
develop a systematic "crosswalk" designed to compare 
the specifier ways each survey asked its training ques- 
tions. 1 To facilitate this aspect of our analysis we bor- 



rowed technology from an unusual source, children's 
"flipbooks," which allow the young to assemble differ- 
ent animals or stories by changing the order in which 
the different parts are revealed. The resulting ability 
to, quite literally, lay out the different surveys side by 
side in any combination highlighted how often the sur- 
veys differed linguistically — as well as how often they 
seemingly asked the same question and still elicited 
substantially different responses. 

We also developed a new way to present graphically 
the statistical differences between surveys. Using ba- 
sic logistic procedures for estimating the odds that a 
specific group of respondents would report having re- 
ceived training, we developed what we came to call 
"odds trees" that showed us at a glance both the gener- 
al estimate of reported training and its distribution 
among groups defined by levels of education, income, 
age, and gender. 

Finally, we used this analytic architecture — both the 
linguistic crosswalks and the odds trees — to explore 
three hypotheses that might explain the contradictory 
estimates of work-related training generated by more 
than a decade of national surveys. 

1. The first, and most stark, hypothesis was simply 
that the surveys were so flawed that they did not 
warrant further analysis. Whether the discrepan- 
cies were caused by linguistic problems inherent 
in the questions, by the order of the questions and 
context in which they were asked, or by the 
differences in the people queried, we speculated 
about the possibility that due to their actual 
construction the instruments themselves were the 
principal cause of the reported variances in 
training rates. 

2. Second, we asked whether there was something 
inherently "slippery" about the nature, and hence 
the meaning and definition, of training that makes 
consistent measurement unlikely. While train- 
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ing's proponents find it easy, and almost trivial, to 
define what they mean by a structured or formal 
training program, is it possible that the beneficia- 
ries of those efforts have a less clear picture of 
how participation in a specific program contribut- 
ed to what they needed to know to work better and 
smarter? 

3. Third, we returned to that old intuition that 
Americans believe in education while down- 
playing, even demeaning, the importance of 
training. In the American context education is 
associated principally with credentials. Ameri- 
cans have a better sense of what they have earned 
in terms of degrees and certificates than what they 
have learned in terms of either knowledge or 
skills. One of the principal characteristics of 



work-related training in the United States is its 
remarkable detachment from credentials. While 
educated citizens, quite literally, have their 
degrees to display, trained workers have little to 
show others — or themselves — to document their 
proficiency. We came to ask: Is it possible that 
both the result and process of training is so 
ephemeral when compared with the result and 
process of earning a high school or college degree 
that it renders the former event unremarkable and 
thus not remembered? Was it possible that 
training has so little intrinsic meaning in the 
American context that the respondents to the 
national surveys relied on the context of the 
training question — rather than their own sense of 
having been trained or not — to form their answers? 



A Catalog of Artifacts 



We begin our story with a description of the surveys 
themselves, 25 separate administrations from 1973 to 
1991. We first looked at the May Triennial Adult 
Education Supplement to the Current Population 
Survey (CPS), which asked questions about the edu- 
cational courses, including formal job training courses, 
taken by adults in CPS's national probability sample of 
approximately 56,000 households. The sample for this 
instrument is drawn to maximize the reliability of the 
survey's estimates of labor force characteristics. While 
comparable data for the May Triennial Adult Education 
Supplement survey were collected as far back as 1969, 
we only examined the years 1981 and 1984, since only 
these administrations included relevant income data. 2 
/ 



Periodically, the CPS has been used to mount spe- 
cial surveys on a particular topic. In January 1983 and 
again in January 1991 the CPS included a specialized 
survey on work-related training. While the 1983 CPS 
Training Supplement increasingly has been utilized to 
analyze the scale and scope of work-related training, 
the public use file containing the January 1991 data 
only recently has become available. Our research rep- 
resents one of the first efforts to match the 1991 results 
to those of 1983 to understand the changing demand for 
work-related training. 1 

The National Longitudinal Survey of Youth 
(NLS-Y) is one of five longitudinal studies adminis- 
tered by the Bureau of Labor Statistics and managed by 

7 



WORKING 3 PAPERS 



the Center for Human Resource Research at The Ohio 
State University. 1 These studies are collectively known 
as the National Longitudinal Surveys of Labor Market 
Experience.' The NLS-Y is a national probability sam- 
ple of 12,686 men and women who were 14 to 21 in 
1979 when they were first interviewed. The most re- 
cent year for which we analyzed data for this project is 
1988. Among the core set of questions asked are items 
covering labor force status, job information, and train- 
ing. 

The Survey of Income and Program Participa- 
tion (SIPP) is a nationwide survey designed to provide 
comprehensive information on the economic situation ' 
of^ouseholds and persons in the United States. The 
first SIPP survey of 20,000 households was conducted 
in October 1983 and is referred to as the 1984 panel. 0 
Respondents were interviewed once every four months 
for approximately two-and-one-half years. A new sam- 
ple panel of approximately 12,000 households is intro- 
duced every year. SIPP collects information on 
income, participation in government transfer programs, 
labor force status, and other topics on a regular basis. 
Various modules addressing topics of special interest 
are administered throughout the two-and-one-half-year 
period. One of these topical modules asky about the 
education and training histories of the respondents. 



The National Longitudinal Survey of High 
School Class of 1972 (NLS-72) is the first major 
study of the National Education Longitudinal Studies 
program of the National Center for Educational Statis- 
tics (NCES).' Tfhe original sample consisted of 22,652 
high school seniors in the spring of 1972. In 1986 a 
questionnaire was mailed to a subsample of 14,489 
members of the original sample. This subset provides 
the most recent data available on this cohort. Training- 
relevant questions were asked during every administra- 
tion of the NLS-72 from the first follow-up and on. 

High School and Beyond (HS&B) was the second 
major longitudinal study undertaken by NCES. 8 The 
base-year survey of HS&B included a 1980 cohort of 
high school seniors who were comparable to the 1972 
cohort. At the same time, a sophomore cohort was in- 
cluded to provide data on the educational choices made 
between the sophomore and senior years. Over 30,000 
sophomores and 28,000 seniors provided base-year 
questionnaire data. The third follow-up of H5&B took 
place in 1986 and remains the most recent data avail- 
able. This questionnaire contains data on 13,425 re- 
spondents from the sophomore cohort and 10,536 
respondents from the senior cohort. Questions about 
training were asked of the senior cohort on all follow-up 
surveys; for the sophomore cohort, training-relevant 
questions were asked on the second and third follow-up 
surveys. 
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The Fundamental Problem 



In attempting to draw consistent estimates of train- 
ing from this wealth of national survey data, research- 
ers have had to confront a host of problems stemming 
largely from the nature and purposes of the instruments 
themselves. Most of the surveys derive from attempts 
to measure something other than training — and in that 
sense the questions on work-related training are pe- 
ripheral to the core items motivating the administration 
of the survey. Not surprisingly, the specific syntax of 
the training items, as well as the context in which the 
training questions are asked, varies across surveys. 
Because the surveys have not been constructed to allow 
the straightforward development of a training measure, 
their use for purposes of estimating the incidence of 
training inevitably requires the researcher to invest 
substantial time and effort in culling information from a 
particular data source. As a result, most researchers 
become familiar with a single data set of choice, im- 
plicitly recognizing that the task of mastering the dis- 
parate sources that do exist is simply too expensive. 9 

The result is the confusion of estimates that confront 
policy makers, who try to approximate the United 
States' current investment in work-related training in 
order to gauge how much more (or less) the nation 
ought to be investing in the educational quality of its 
workforce. Estimates of the cost of training, for exam- 
ple, range from 2 billion dollars annually to 200 or 
more billion dollars per year (Zemsky and Meyerson 
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1981; Eurich 1990) The number of people who report- 
ed receiving work-related education or training has 
been estimated variously at 20 percent, 55 percent, and 
70 percent of the working population — depending on 
the data source used and its particular definition of 
training. Interestingly, although there is little agree- 
ment on the incidence of work-related training, most 
research finds that the most likely recipients tend to be 
male, while, well-educated, and between the ages of 25 
and 44. 

For our analysis, the staff of the National Center on 
the Educational Quality of the Workforce (EQW) de- 
rived new estimates, of work-related training from each 
of the surveys described above. The results of the ba- 
sic tabulation — along with the text of the training ques- 
tions themselves — are presented below in Table 1. 

The task set for the EQW work-team was the devel- 
opment of a crosswalk between and among these na- 
tional surveys to facilitate a better understanding of the 
causes of the variations in training rates. We note at 
the outset that the greatest similarity among these sur- 
veys is their form. Each instrument focuses on the 
experiences of individuals — rather than the experienc- 
es of either firms or the suppliers of work-related train- 
ing — as reported by the individuals themselves or by a 
member of their household. Since the federally-spon- 
sored individual and household surveys included in the 
EQW crosswalk are national probability samples of 
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Data Set 


Population 


Training Question 


Percent 

Responding 

Affirmatively 


NLS-72 as of 
the 1986 
Follow-up 


All workers with 
reported income 
in 1986 who 
were high school 
seniors in 1972 


Between the time you left high 
school (or the last interview) and 
October 1973 (or the current year), 
have you participated in any 
program such as on-the-job- 
training, registered 
apprenticeships, manpower 
training, personal enrichment, or 
correspondence courses? Do not 
include Armed Forces training 
programs, or regular school and 
college programs. 

or 

Not including on-the-job training, 
did you receive formal instruction 
to do this kind of work? (asked for 
the 2nd through 4th follow-up 
surveys. ) 

or 

Considering the most recent full- 
time job you have held, did you 
receive or participate in any type of 
employer provided training 
benefits or training programs? 
(Asked for the 5th follow-up only.) 


86% 
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Table 1 (continued) 



1 Data Set 1 


Population j ' 


Training Question 


Percent | 
Responding 
Affirmatively J 


High School & J 
1 Beyond - as of I 

the 1986 
J Follow-Up 


All workers with I 
reported income I 
in 1986 who 
were high school J 
seniors or 1 
sophomores in 1 
1980 


Between the time you left high 
school (or the previous interview) and 
the end of February 1982 (or the 
February of the current year) have 
you participated in any program 
such as registered apprenticeships 
or manpower training programs? 
Do not include regular school or 
college programs or armed forces 
training programs. 

or 

Considering the most recent full- 
time job you have held, which 
type(s) of employer-provided 
training benefit(s) or training 
program(s) did you receive or 
participate in? 

or 

Not including on-the-job training 
or employer provided training, 
have you received formal training 
to do your current (or most recent) 
job? 


75% 


1 January 1991 
I CPS 


1 All workers aged 
1 22 to 65 with 
I reported income 


1 Did you need specific skills or 
training to obtain your current 
(last) job? 

or 

1 Since you obtained your present 
job did you take any training to 
1 improve your skills? 


70% 1U 


January 1983 
CPS 


j All workers aged 
I 22 to 65 with 
1 reported income 


1 Did you need specific skills or 
1 training to obtain your current 
(last) job? 

1 01 

Since you obtained your present 

| job did you take any training to 
1 improve your skills? 


66% 



10 The rate excludes non-respondents. The public use file for the January 1991 CPS had about 1 in 5 
nonrespondents for training questions. 
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Tablet (continued) 



Data Set 


Population 


Training Question 


Percent 

.Responding 

Affirmatively 


NLS Youth 
Cohort as of 
1988 


All workers 23- 
30 as of January 
1, 1988 with 
reported income 


Since a year ago, have you received 
skills training from a government 
sponsored program such as CETA, 
the Job Corps, or any of these other 
government-sponsored programs 
where young people who are not 
attending regular school are 
provided with skills training? 
or 

Since (last interview), have you 
received training from any (other) 
source, such as the kinds of places 
listed on this card? For example, 
training in a business college, 
nurse's program, an apprenticeship 
program, voc-tech institute, or any 
of these other kinds of sources? 

Versions of these questions were asked 
at each interview from 1979 through 
1988 


59% 


SIPP 1987 
Wave 2 


All workers aged 
22 to 65 with 
reported 
earnings 


Has ever received training 
designed to help find a job, 
improve job skills, or learn a new 
job? 


29% 


SIPP 1986 
Wave 2 


All workers aged 
22 to 65 with 
reported 
earnings 


Has " ever received training 
designed to help find a job, 
improve job skills, or learn a new 
job? 


29% 


SIPP 1984 
Wave 3 


All workers aged 
22 to 65 with 
reported 
earnings 


Has ever received training 
designed to help find a job, 
improve job skills, or learn a new 
job? 


24% 


May 1984 CPS 


All workers aged 
22 to 65 with 
reported income 


(Excluding full-time attendance in 
school), have you taken parjt in any 
organized adult education courses 
or activities? 


19% 


May 1981 CPS 


All workers aged 
22 to 65 with 
reported income 


(Excluding full-time attendance in 
school), have you taken part in any 
organized adult education courses 
or activities? ' 


17% 

/ 
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individuals in the workforce, they have the further ad- 
vantage of combining within a single instrument ques- 
tions about work-related training supplied Dy 
employers, about both formal and informal on-the-job 
training, and about training gained through regular 
schooling and as an adult learner. Finally, household 
and individual surveys allow a full representation of 



workers employed by small businesses, which are often 
undenvpresented in national surveys of firms. What 
none of these national surveys supplies, however, are 
adequate data on the employing firms themselves— an 
oversight that should to be corrected in future adminis- 
trations. 



The Text-Based Crosswalk 



Our first step toward understanding the differences 
between and among these national household and indi- 
vidual surveys was to develop a strategy for identifying 
the extent to which language atone might account for 
the training estimates derived for any pair of surveys. 
Our answer was the invention of a survey catalog— what 
we would eventually call the Crosswalk of National 
Data Sets Focusing on Worker Training, which lists the 
specific questions asked and, where appropriate, the 
basic categories of allowed responses to questions 
about types of training and sources of payment. We 
also included within our catalog the available popula- 
tion characteristics, the structure of the survev in terms 
of skip patterns and question contexts, and the sam- 
pling frame that the survey employed. 

The Crossivalk is designed to allow researchers to 
compare questions and options both among all the sur- 
veys and within the same survey across different years 
of administration. The Crosswalk is a double-sided 



book which, when opened, resembles two identical 
books placed side-by-side with mirror-image pages. 
Figure 1 provides a schematic representation of the 
Crosswalk, while Figure.2 gives a sample entry for one 
survey-in this case the 1984 S1PP. Each of the tables 
appears in both halves of the crosswalk, allowing for 
the comparison of any two training questions. The 
layout covers the four broad areas necessary to under- 
stand why various estimates of worker training might 
differ: 

1 . Questions and options pertaining to work-related 
training. 

2. Skip patterns that determine the specific popula- 
tion that is asked to respond to each question 
about training. 

3. Population characteristics available for grouping 

respondents. 

4. Organizational characteristics including the 
sponsoring agency and the survey's length and 
frequency of administration. 
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can vary substantially in the types of options offered to Forces Training Programs" are always listed in the 

a respondent, we "crosswalked" these options by reor- same row.) 
dering them to facilitate a comparison, so that similar 



Beyond Logical Differences 



No one should expect the surveys included in the 
Crosswalk to yield identical results, largely due to the 
structure of the surveys themselves. The NCES data 
sets, for example, are the result of the separate sam- 
pling of specific high school students over time. As 
such, they logically will report a different incidence of 
training than a sample of the population as a whole. 
Similarly, the focus of the 1981 and 1984 May Trienni- 
al Adult Education Supplements to the CPS will in- 
volve a different definition of work-related education 
and training than the one employed in the special 1983 
and 1991 January Training Supplements, which were 
designed specifically to measure the level of work- 
related training in the United States. Even nearly iden- 
tical surveys, such as the January 1983 and January 
1991 CPS special training supplements, can yield dif- 
ferent results, either because of sampling variations or, 
more likely, the passage of time. These variations in 
sample, language, and purpose among the surveys are 
readily identified using EQW's Crosswalk (sec pages 
12-15). Other differences, however; are not as easily 



reconciled. Take, for example, a comparison between 
the January 1983 administration of the CPS Training 
Supplement and the first administration of the Survey 
of Income and Program Participants. 

This January Training Supplement attempted to cap- 
ture—in a deceptively simple pair of questions— the 
full range of activities that workers might identify as 
training. One question read: 

Did you need special skills or training to 
obtain your current job? 
Among employed persons aged 22 to 65 years, 56 per- 
cent answered affirmatively to this question focusing on 
qualifying training. Another asked: 

Since you obtained your present job did you 
take any training to improve your skills? 
Among employed persons aged 22 to 65 years, 38 
percent answered "yes" to this question focusing on 
skills improvement. In all, 66 percent answered "yes" 
to at least one of the two questions. Still, we were 
struck by the fact that 34 percent said they needed 
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neither skills nor training to obtain or keep their 
present jo^s. It made us wonder. 

This second survey — or rather, set of surveys — be- 
gan less than a year later. The Survey of Ijicome and 
Program Participation (SIPP) was commissioned to 
provide policy makers with information to study govern- 
ment tax and transfer programs, estimate future pro- 
gram costs and coverage, and assess the effects of 
proposed policy changes. 

Designed and conducted by the same organizations 
responsible for the 1983 CPS January Training Supple- 
ment, the SIPP questioned people about training in 
virtually the same way: 

Has ■_ ever received training designed 

to help find a job, improve job skills, or 
learn a new job? 
Surprisingly, just 24 percent of employed persons aged 
22-65 years answered "yes." 



Curious about the linguistic tilt of the surveys, we 
began asking members of the staff, visitors to the Cen- 
ter, and even the audience of a Washington Public 
Policy Seminar to guess which of the two sets of ques-s 
tions would be more likely to elicit a larger positive 
response. Almost uniformly, the SIPP question was 
viewed as broader, more inclusive, and more likely to 
engender positive answers. 

To resolve the differences between these two well- 
constructed national surveys with well-administered 
protocols, extensively field-tested survey questions, 
and practically the same sampling frame, we set about 
constructing a statistical companion to complement the 
linguistic comparison presented in our crosswalk. 
Adopting the historian's guise, we approached the two 
surveys as sets of transactions. What were we being 
told about the sample population? About the survey 
questions? About the context for interpreting educa- 
tion and training? 



Comparison of CPS 1983 & SIPP 1984 



CPS 1983: 

58, 000 households 
10-12 minutes per person 



SIPP 1984: 

20, 000 households 
25-30 minutes per person 
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Figure 1 

Design for A Crosswalk of National Data Sots Focusing on Worker Training 
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Figure 2 

Crosswalk Right-Hand Side Entry for SIPP 1984 
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DEMOGRAPHIC QUESTIONS AND OPTIONS 
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The Sampled Population 



Our first task was to develop a sufficiently broad 
scheme for describing the population samples respond- 
ing to each survey in the crosswalk. What we sought 
was a set of definitions that would allow the ready com- 
parison of subsets based on gender, education, income, 
and age — even though the surveys used slightly differ- 
ent questions to chart the population characteristics of 
their samples. Most, but not all, of these discrepancies 
involved how the different surveys asked respondents 
to report their income. 

For both practical and theoretical reasons, we elect- 
ed to characterize the sampled populations using a 
series of categorical variables, which provided a sim- 
plification that made data comparisons across surveys 
possible. As it turned out, however, there were also 
important substantive reasons for looking at the data in 
this way, largely because we did not want to assume 
a priori the nature of the relationship between training 
and our four population characteristics: gender, educa- 
tion, income, and age. 

The analytic scheme we adopted placed each work- 
ing respondent from each sampled population into one 
of 128 "buckets" based on four categories describing 
the respondents' education, four age cohorts, four in- 
come bands, and two genders. The 128 (4x4x4x2 = 
128) mutually exclusive and exhaustive categories thus 
defined our analytic structure. 

20 



Education 

Although researchers frequently use years of school- 
ing to explain differences in income, we believed that 
in a nation in which decrees, rather than years of 
schooling, are the better predictor of labor market ex- 
perience, it made sense to focus on educational creden- 
tials. Accordingly we placed" each respondent into one 
of four categories of educational attainment: 

• No High School Degree 

• High School Degree Only 

• Some College but not a Bachelor's Degree 

• A Bachelor's Degree or Better 

Age 

Ordinarily age is treated as a continuous variable. 
Here too, we wanted to allow for the possibility that age 
introduced discontinuities into the distribution of work- 
related training. We separated respondents into one of 
four eleven-year age-bands, classifying them by their 
age in the year in which the survey was administered. 
.While there is a degree of arbitrariness to our defini- 
tion of age-band categories for defining cohorts, we 
found the following classifications to be useful in inter- 
preting the data: 

• Respondents aged 22-32 

• Respondents aged 33-43 

.- 

• Respondents aged 44-54 

• Respondents aged 55-65 

During our initial examination of the data, it became 
clear to us that the reported income for younger and 
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older cohorts were biased downward— attributable, we 
believed, to the part-time work status of many students 
and semi-retired adults. Accordingly we limited our 
analysis to those respondents between the ages of 22 
and 65 at the time of the survey. 

Income 

Given the different ways in which the surveys report- 
ed the income of respondents, it proved impossible to 



make comparisons treating income as a continuous 
variable. Moreover, in the absence of any prior knowl- 
edge concerning a specific relationship between in- 
come and training, we were hesitant to assume that 
income acted in a linear fashion. Our solution was to 
categorize respondents based on the income quartiles 
into which they fell. 10 



An Architecture for Analyzing the Structure of Responses 



Using the above definitions along with a two- 
category variable for gender, we addressed the issue of 
the similarities and differences among the sampled 
populations. We knew that the*January 1983 CPS and 
the S1PP 1984 panels should have been similar be- 
cause they are separated by only a brief period of time. 
In addition, both were household surveys of the non- 
institutionalized United States population and both 
were administered by the Bureau of Labor Statistics 
using nearly identical sampling frames. 

For any sampled population, each of the 128 sepa- 
rate "buckets" defined above will account for a specific 
percentage of the total population. The question we 
asked was simply, "Are the corresponding buckets for 
any two surveys roughly the same size?" More specifi- 
cally, we focused on the square roots of these bucket 
percentages, since they, unlike the raw tallies, yield a 
distribution that is approximately normal. 



Figure 3 shows a comparison between the square 
roots of the population percents for each bucket for the 
January 1983 CPS and the 1984 SIPP. The solid line 
marks the 45° slope along which the points in the graph 
would lie if each category represented exactly the same 
population percent in both surveys. While there is 
some variation between the two surveys, the sampled 
populations look remarkably similar. The product- 
moment correlation between the two measures is 0.97. 
Insofar as our analytic framework taps the relationship 
between gender, education, income, and age, we con- 
cluded that the January 1983 CPS and 1984 SIPP drew 
their samples from the same population. 

Thus far, we have eliminated two potential causes for 
the significant differences between the estimates of 
training derived from the January 1983 CPS and the 
1984 SIPP: linguistic discrepancies and population 
differences. Essentially, the two surveys asked highly 
similar questions of two, nearly identical population 
samples. 
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Figure 3 

Square Roots of Population Percents 




ooooooooooo 



r=0.973 CPS January 1 983 



To explore further the underlying differences be- 
tween the two surveys, we developed a modeling strate- 
gy that allowed us to compare the relative propensity of 
different groups within the population to report that 
they had received training. As an analytic companion 
to our linguistic crosswalk, the staff of the National 
Center for the Educational Quality of the Workforce 
compiled the results of this modeling in two volumes of 
tabular data entitled, Statistical Companions to the 
Crosswalk of National Data Sets Focusing on Worker 
Training, 1 1 



Use of a Logit Model to Produce an Odds-Tree Representation 



The dependent variable in our model became the 
odds that respondents with a particular combination of 
characteristics would report that they received training. 
Again, those characteristics reflected our categories of 
education, age, income, and gender. Two consider- 
ations led us to use a logistic analysis to estimate the 
odds of receiving training. The question we now posed 
was not how much training that respondents reported {a 
variable outcome), but whether the respondent did or 
did not report the receipt of training (a discrete out- 



ERIC 



WORKING 



II 



come). A logistic analysis yields parameters in the 
form of odds that are readily interpretable as well as 
lending themselves to a concise graphical presentation. 

The rationale behind the logistic analysis is rather 
straight-forward. For each of the t£8 analytic buckets 
in our model, there is a probability P i that individual 
members of the group will respond positively to a given 
training question. The value of P i can vary from 0 to 
1. The expression jj gives the relative likelihood of 
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someone in bucket i reporting that they had received 

training. If the probability of receiving training is 0.5, 

then for every person who received training there is one 

person who did not, and the relative likelihood, or 

odds, of reporting that one had received training is 1 to 

1. If the probability of a given cell is .75, then the 

relative likelihood is 3 to 1. Similarly if the probability 

p 

is .25, the odds would be 1 to 3. The term - ' can 

f P 1 . ! ' R . 
vary from 0 to oo. The term Ln 1 ^ ± I , known is the 

logit, is the log of the odds of the event occurring; in 
this case it is the receipt of training. The logit can vary 
from -oo to oo. h is this value that is predicted when we 
perform a logistic analysis. 

If we let Ln f 1 =y , then our model takes the form 
of Y i m /JX; + F f , ' where /? is a vector of parameters 
given by a maximum-likelihood estimator of the logistic 
distribution, X i is a vector of variables (in our case 
dummy variables for cohort, income, education, and 
gender), and € i is a stochastic error term which should 
be normally distributed. The vector /? is easily inter- 
preted if the base for the logarithms is chosen carefully. 
For our analysis we chose a base-2 logarithm so that 
changes in Y i are easily interpreted in terms of powers 
of two. A unit change in Y i represents a doubling (or , 
halving) of the odds of receiving training, a change of 
two units in Y t would represent a quadrupling (or quar- 
tering) of the 'Kids, a change of three units would repre- 
sent changes of a factor of eight, and so on. 12 

Along with the logit analysis, we designed a graphic 
io presentation that would allow us to observe the varia- 
tions in the basic incidence of training among our na- 
tional surveys. 13 The result — what we have come to 
call an "odds tree" — consists of an intercept that repre- 
sents the basic odds for receiving training for all the 
categories in the model and of adjacent bars that repre- 
sent the degree to which the odds for respondents in 
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each category differed from the basic odds. The inter- 
cept serves as the "trunk" of the odds tree, while the 
bar graphs resemble "branches." We have found that 
these odds trees greatly facilitate a discussion of train- 
ing numbers; not only can we observe a vertical com- 
parison of individual odds compared to the intercept 
(which also establishes monotonic or curvilinear rela- 
tionships), but also we can sum up tKe values of bars 
from different categories horizontally, allowing us to 
estimate training for someone who possesses character- 
istics of more than one category (a woman with a bache- 
lor's degree whose income falls in the first quartile, for 
example). 

Figure 4 illustrates this graphical form. In this case 
we use the odds tree to present the results of the logit 
analysis for the January 1983 CPS. The solid vertical 
line represents the unweighted average of all the cells 
in our analytic space and in this case intersects the 
horizontal axis just above or to the left of the "2 to 1" 
mark. Averaged over all the cells, for every 2 respon- 
dents who replied they had received training, 1 replied 
that he or she had not. 

Several other points concerning the responses \o the 
January 1983 CPS questions on training can be ob- 
served in the above graph. First, we note that both 
education and income are monotonically related to 
training. Having a bachelor's degree or better more 
than doubles the odds of responding that training has 
been received, relative to the unweighted average. Not 
having a high school diploma reduces those odds by 
more than 50 percent. Those without high school di- 
plomas have odds of receiving training that are less 
than 20 percent the odds of respondents with at least a 
bachelor's degree. Respondents with only a high 
school diploma and only some college fall between 
these two extremes. 
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Similarly, being in the highest income quartile in- 
creases the average odds by a factor slightly less than 2. 
Being in the lowest income quartile cuts by more than 
half the odds of a respondent saying that they had re- 
ceived training relative to the average. Everything else 
being equal, women have greater odds than men of 
responding that they received training. Averaged over 
the sixty-four possible categories, for every 5 males who 
responded that they received training, another 3 respond- 
ed that they did not; for every 5 females who responded 
that they received training, 3 responded that they did not. 

Figure 4 refers to the union of two separate ques- 
tions asking about the training received by respon- 
dents: training that qualified the respondent for a 
specific job and training that provided skill improve- 
ment. Figures 5 and 6 present the odds trees for each 
question separately. 



Several differences are worth noting when comparing 
the logit analyses of the separate questions, Figures 5 
and 6, with the analysis presented in Figure 4. The 
most obvious is the shift of the "trunks" in Figures 5 
and 6 to the left, detailing what we already knew: 
namely, that data for Figure 4 is the union of the data 
for Figures 5 and 6. When the question becomes "Did 
you receive any training, either qualifying or skill im- 
provement?" there is a clearer income effect; the 
"branches" on Figure 4 are clearly longer than those on 
Figures 5 and 6. 

More apparent, however, are the basic similarities 
among the three graphs — the odds trees, in fact, look 
alike. What we conclude is that, in their relation to 
gender, income, age, and education, there is little ap- 
preciable difference between qualifying and skill im- 
provement training as measured by the January 1983 CPS. 



Figure 4 

CPS Jan, 1983 (Any Training) • Results of Logit Analysis 
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Figure 5 

CPS Jan, 1983 (Qualifying Training) • Results of Logit Analysis 
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Figure 6 

CPS Jan, 1983 (Skill Improvemnt Training) • Results of Logit Analysis 
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Odds Tree and Estimated Model: January 1991 CPS 



Thus far our analysis has offered interesting ways to 
compare and contrast different elements of the same 
survey. We turn next to the usefulness of our architec- 
ture in discerning differences and similarities between 
surveys. Figure 7 presents the results of a logit analy- 
sis performed on the responses to the two training ques- 
tions from the January 1991 CPS that were identical to 
the training questions asked in the January 1983 CPS. 
Except for the slight shift of the trunk to the right that 
reflects the fact that a higher proportion of the respon- 
dents reported the receipt of training in 1991 than they 
did in 1983, the resulting odds trees again are remark- 
ably similar. It is also the case that the odds trees (not 



shown) for the separate questions detailing qualifying 
and skill improvement training from the January 1991 
-CPS appear exactly like their counterparts from the 
January 1983 survey (Figures 5 and 6, above). 

We also asked: "If we use weights derived from 
the logit model for the January 1983 CPS, how well 
can we predict the'results of the January 1991 CPS?" 
Figure 8 presents that forecast. The parallel tracking 
of the actual data and the predicted results help per- 
suade us that the increase in training rates between 
1983 and 1991 c an not be attributed to differences in 
gender, age, income, or education. 
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Figure 7 

CPS Jan, 1991 (Any Training) • Results of Logit Analysis 
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Figure 8 

1991 Observations vs. 1983 Predictions 
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Survey Similarities 



Based on these comparisons, our conclusion was that 
the 1991 survey was a valid follow-up to the 1983 sur- 
vey in terms of the underlying structure of responses: 
training increases monotonically with education and 
income; women, when controlling for income, age, and 
education, are more likely to receive training than men; 
and age, once income, gender, and education are con- 
trolled for, has little consistent impact on who reports 
the receipt of training. We concluded that the right- 



ward shift of the trunk for 1991 indicated a measurable 
increase in the receipt of training over an eight-year 
period. 

The comparison of the two CPS surveys also increased 
our confidence in the analytic architecture we had de- 
veloped, because we expected to find similarities and 
did so. The odds trees provided an easily interpretable 
graphic for identifying the common structure underly- 
ing the responses to survey training questions. 



Odds Tree and Estimated Model: SIPP 1984 



We turn next to the 1984 panel of the Survey of In- 
come and Program Participation (SIPP) — the survey 
with one of the lowest affirmative responses to the training 
question — to ask, "Can our architecture help explain 
differences as well as validate similarities between t 
survey results?" The odds tree derived from the logit 
model is displayed below in Figure 9. 

The first difference between the SIPP odds tree and 
those for the two CPS surveys (Figures 4 and 7 above) 
is the expected shift of the trunk to the left. On aver- 
age, for every person who responded in the SIPP instru- 



ment that they had received training, almost four peo- 
ple responded that they had not. This ratio represents 
the inverse of the odds reported in the 1983 CPS survey. 

What is more. important, however, is the fact that the 
SIPP-CPS differences are not limited to the absolute 
magnitude of the response rates. The general pattern of 
the responses — that is, the shape of each odds tree in 
terms of its branches — is fundamentally different. While 
the relationship between income and training is mono- 
tonic in the SIPP data, the range of variation between 
the top and bottom income quartiles is much smaller 
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than in the CPS. In the January 1991 CPS survey, the 
odds of receiving some type of training increased by * 
almost a factor of seven when moving from the bottom 
to the top income quartile. In the 1983 version of that 
survey the odds increased about six-and-one-half times 
when comparing the top to the bottom income quartile. 
In the 1984 SIPP panel, on the other hand, the increase 
in odds when moving from the bottom to the top income 
quartile is a factor of one-and-one-half. In both of the 
CPS surveys there is a moderate gender effect: females 
tend to be more likely to respond that they had received 
training than comparable men. In the SIPP, on the other 
hand, the effect is slight and in the opposite direction. 

Perhaps the most striking difference between the two 
surveys has to do with the relationship between levels 
of education and propensity to respond that training has 
been received. In the CPS there is a strong and mono- 
tonic relationship between education and training re- 



sponses. In general, those with more advanced degrees 
also reported they had received more training; those 
with less education reported less training. In the SIPP 
we see a. curvilinear relationship. Those without a high 
school diploma are least likely to say they had received 
training (as in the CPS). In contrast with the CPS, SIPP 
respondents with only a high school diploma, all else 
being equal, are the most likely to say they had been 
recipients of training. Those with some college, but not 
a bachelor's degree are the second most likely group, 
and those with a bachelor's, degree or more rank just 
above those with no high school diploma. 

These results are not unique to the 1984 SIPP panel. 
Just as the CPS training supplements showed a consis- 
tent underlying structure, the different SIPP panels 
also show a similar — though not identical — structure. 
Figures 10 and 1 1 display the results of the logit model 
for the 1986 and 1987 SIPP panels respectively. 



Figure 9 

SIPP 1984, Wave 3 • Results of Logit Analysis 
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Figure 10 

SIPP 1986, Wave 2 • Results of Logit Analysis 
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Figure 11 

SIPP 1987, Wave 2 • Results of Logit Analysis 
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Goodness of Fit 



Choosing a logit model as the basis for our analytic 
architecture allowed a ready comparison among differ- 
ent data sets in which the dependent variable was Bool- 
ean (reported receiving training; did not report receiving 
training) and the independent variables were group 
characteristics that were not necessarily ordered. It is 
a procedure that allowed us to test directly for non- 
linear relationships between training and our four pop- 
ulation characteristics: education, income, age, and 
gender. 

Unlike simple regression models, there is no statistic 
analogous to an R 2 associated with the logit analysis — 
the notion of percent variance simply loses its meaning 
when predicting a Boolean dependent variable. It is 
still possible, however, to assess a model's capacity to 
predict training rates in a rather intuitive way. Logit 
analyses do provide a series of measures, which include 
chi-square statistics and their associated probabilities, 
a goodness of fit measure (c), and association of pre- 
dicted probabilities and observations in the form con- 
cordant and discordant pairs. For us, it is the last 



measure that has proved to be the most satisfying way 
to gauge whether one logit model or another better fits 
the data. The concordant pair analysis examines all the 
possible pairs of respondents with different responses. 
A pair is concordant if the respondent who reported 
receiving training had higher predicted odds of doing 
so than the respondent who did not report receiving 
training. A pair is discordant if the respondent with 
training had lower predicted odds than the respondent 
without training. If the predicted odds were identical 
then the pair is a tie. 

Table 2 presents the percent of concordant pairs for 
the logit models presented in Figures 4 and 7, January 
1983 and January 1991 CPS respectively, and Figure 9, 
the 1984 SIPP panel. It is clear that the CPS models 
provide substantially better fits than the SIPP model. 
We concluded that for the SIPP there were likely vari- 
ables other than education, age, income, aiid gender 
that helped to explain who reported the receipt of train- 
ing — variables that did not play the same role in ex- 
plaining responses to the two CPS surveys. 



Table 2 
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Survey Percent Pairs Concordant 

CPS January 1 983 (Any Training) 73% 
CPS January 1991 (Any Training) 76% 
SIPP 1984 (Wave 3) 59% 
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Analysis of Sample and Textual Differences 



We return, then, to the central question with which 
we began our discussion. Why should two national 
surveys administered roughly at the same time that ask 
basically the same questions of nearly identical popula- 
tion samples yield such widely divergent responses? In 
some sense, the mystery is deepened by the discovery 
that the structure of the responses are also different. In 
the survey that yielded the higher estimate of the inci- 
dence of training, there was a clear, monotonia relation- 
ship between educational credentials and the reported 
receipt of training. In the other, the relationship be- 
tween educational credentials and training was both 
less certain and curvilinear. If neither sample differ- 
ences nor question syntax are the answer, what other 
elements of our first hypothesis might suggest a reason- 
able explanation? 

Survey Length 

The CPS training supplements are added onto a 
standard Commerce Department household survey that 
is administered monthly. The CPS itself has the well- 
defined purpose of providing data on labor force activi- 
ty. Comprehensive data are collected on the 
employment status, occupation, and industry of the 
respondents. The survey is relatively short, taking onlv 
10 to 12 minutes per subject. The S1PP is a more 
broadly based survey with the purpose of providing 
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"comprehensive information on the economic resources 
of the American people and on how public transfer and 
tax programs affect their financial circumstances" 
(Kasprzyk 1988, 1). . 

A more detailed survey than the CPS, SIPP requires 
almost 25 to 30 minutes per subject to administer, with 
the training questions asked towards the end of the 
instrument. One possible source of the discrepancies 
between CPS and SIPP simply may be the length of the 
latter, which could induce a significant number of re- 
spondents to shorten the time of the interv iew by an- 
swering trigger questions negatively in order to avoid 
more detailed follow-up questions. 

Question Context 

The fatigue factor, in itself, cannot explain the re- 
sults of our logit analysis — unless one assumes it is the 
more educated who want and know how to shorten the 
survey by answering the "trigger" questions negatively. 
A more plausible explanation lies in the differing na- 
ture and order of the surveys themselves. 

In the CPS surveys, the training supplement immedi- 
ately follows a series of queries asking about the re- 
spondents' jobs and their work. The SIPP topical 
modules focusing on training immediately follow a sec* 
tion of questions asking about the Food Stamp program, 
SSI, dnd participation in other government-sponsored 
income maintenance programs. Our hypothesis is that 
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a significant number of respondents who were not par- 
ticipants in these programs reported they had not re* 
ceived training simply because, in their minds, the 
survey associated training with participation in a public 
welfare program. The curvilinear relationship between 
training and education in the SIPP responses becomes 
more explicable under this scenario: the more educat- 
ed the respondent (and the less likely to benefit from 
public welfare), the less likely he or she will respond 
affirmatively to the SIPP training question. On the 
other hand, those without high school diplomas — 
whether or not they participated in a public welfare 
program — did in fact receive little training, however 
that concept was understood by the respondent. The 
questions that remain to be explained are why income 
monotonically relates to training in both the SIPP and 
CPS surveys and why the relationship is markedly 
stronger in the latter than in the former. 

Training as Mirage 

We turn now to the second of the hypotheses intro- 
duced at the outset of this paper: the context of the 
training question becomes extremely important because 
of the "slippery" definition of training itself. Since 
training, as a concept, may have little intrinsic meaning 
for many respondents, they may infer what the question 
signifies by drawing conclusions from the purpose of 
the survey and the nature of the preceding questions. 
Why do estimates of the incidence of training vary so 
widely? Because there is nothing specific enough to 
measure. It is in this sense that training is something 
of a mirage — visible in the distance, but intangible at 
close proximity. 

There are important policy implications attached to 
such a conclusion. What has emerged over the last 
half-decade is an important consensus among policy 
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makers about the importance of making direct invest- 
ments in The continuing educational quality of the 
workforce. In most proposals, such investments trans- 
late into the creation and funding of new training pro- 
grams for first-time workers as well as those made 
redundant by technological and economic change. 
What the conflicting results of the surveys suggest is 
that even as an opportunity for public investment, 
training may appear more real to the policy maker than 
to the intended beneficiary. Training, as a concept, 
needs to be better understood — and in a real sense, 
owned — by those whom work-related training is expect- 
ed to benefit. The call should not simply be for new- 
initiatives and increased funding, but for a focused 
effort to create a real context for training. 

It is interesting to note that across all the surveys 
included in our analysis, there was remarkable consis- 
tency in the reporting of educational credentials and 
their relation to age, income, and gender. Even to the 
extent that individuals exaggerated their educational 
attainment, they apparently did so in a very consistent 
manner. Neither the length of the survey, the context, 
nor the specific content of the educational questions 
had much impact on the proportion of the sampled pop- 
ulation reporting either graduation from high school or 
receipt of a college degree. 

Education s. Training 

It was this clear and consistent difference in how 
respondents reported their educational attainment as 
opposed to the receipt of work- related training that 
yielded our third and frankly most speculative hypothe 
sis: that the differences across the range of national 
surveys document that old intuition that Americans 
believe in education while down-playing, even demean 
ing, the importance of training. The education commu 
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nity has long focused on the differences between edu- 
cation and training — arguing that the former is specific 
and narrow, while the latter is broad and general. 
Training, as an educational concept, is reduced to a 
focus on vocational skills and on what a worker needs 
to perform a specific job better. Education, on the 
other hand, is portrayed as being about the acquisition 
of knowledge, about learning how to think critically and 
solve problems. 

We suspect that such distinctions betyyeen'education 
and training — if they are in fact real — pale in impor- 
tance to the most obvious distinction between the two: 
in the United States education is principally about 
credentials and training is principally without such 
distinctions. In a modern economy, credentials oonfer 
property rights just as, in an earlier age, land and entry 
into a father's (or uncle's or grandfather's) occupation 
or craft allowed middle-class families to pass on their 
status from father to son. Training in the United States 
is almost wholly without that sense of credentialing. . 
There tre no degrees and few requirements outside of a 
limited number of highly technical occupations and 
skilled crafts. One explanation for why training is 
hard to measure consistently is simply that it is so un- 
like education — not in terms of style or content, but in 
terms of tangibility. Put simply, it may be the absence 
of credentials that makes training such a mirage. 

Beneath the surface, however, there may be a larger 
lesson — perhaps even a deeper structure — that the 
national surveys partially illuminate. When we exam- 
ine all the surveys, two rough groupings emerge: those 
surveys associated with an educational agency, pur- 
pose, or process on the one hand and, on the other, 
those surveys more directly associated with the making 
of cither welfare or economic policy. The three longitu- 
dinal surveys in our set — NLS-72, High School and 



Beyond, and NLS-Y — help to illustrate this point. 
Each survey follows a separate youth cohort: NLS-72 
follows high school seniors who graduated in 1972; 
High School and Beyond follows the high school gradu- 
ating class of 1980; and NLS-Y follows a sample of 
youth who were 14 to 21 in 1979. 

The first of the longitudinal studies, the National 
Center for Educational Statistics' National Longitudinal 
Survey of the High School Class of 1972, has consis- 
tently asked its panel high school seniors to report the 
receipt of work-related education and training in each 
of its follow-ups (1973, 1984, 1976, 1979, and 1986). 
Figure 12 presents the odds tree reporting the cumula- 
tive probability of a respondent reporting the receipt of 
training. Remarkably, each structure is similar to the 
odds trees for the two CPS surveys focusing on train- 
ing— CPS 1983 and CPS 1991. The receipt of training 
increases monotonically with increases in both educa- 
tion and income. For this cohort of high school seniors, 
training was an expected activity, with the odds of re- 
porting its receipt at close to 8 to 1. 

The National Center for Educational Statistics also 
commissioned a parallel longitudinal study that began 
tracking 1980 high school seniors and sophomores. 
This study has come to be known as High School and 
Beyond which, like NLS-72, has consistently asked 
about work-related education and training in each of its 
three follow-ups (1982, 1984, and 1986). Figure 13 
presents the cumulative odds tree for the reported re- 
ceipt of any training. It also resembles the basic CPS 
odds trees, except that those who have earned a BA/BS 
report significantly less training than expected — an 
artifact telling us that in 1986, the year of the last fol- 
low up, most of the sample of college graduates had just 
earned their degrees. 
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Figure 12 

NLS-72 Any Training Received (First 5 Follow-Ups): AH Races • Results of Logit Analysis 
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Figure 13 

HSB Any Training Received (First 3 Follow-Ups): All Races • Results of Logit Analysis 
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The remaining longitudinal study of a youth cohort 
was developed by the Bureau of Labor Statistics largely 
to track the cohort's progress through the labor market. 
Given its focus, it is not surprising that NLS-Y yields 
quite different results. Figure 14 shows the odds tree 
reflecting the report of private training. Here is the 
curvilinear relationship between educational attain- 
ment and the receipt of training that we first observed 
in SIPP 1984. There, is a similar reduction in the over- 
all reporting of such training, although the relationship 
between income bands and reported receipt of training 
is stronger in NLS-Y than in SIPP 1984. 

NLS-Y also asked respondents to report the receipt 
of government training. The odds tree depicting any 
government training, Figure 15, demonstrates how gov- 
ernment-sponsored training has been well-targeted and 
reinforces the often stereotypical portrait of the popula- 
tions such programs are expected to benefit — those 
without educational attainment and those with the low- 
est income. 

What we have come to conclude is not that training 
has no meaning in the American context; rather, it has 
two meanings. Americans think of job training as 
either career advancement for the educated or as a 
second opportunity for the disadvantaged — two stereo- 
types deeply rooted in the policies and practices of the 
last forty years. 

The history of work-related education and training in 
the United States is, in fact, divided — containing two 
streams of development, each with its own definitions 
and values, resulting in quite different policies and 
practices. The first type of work-related education and 
training is an outgrowth of corporate America. The 
customs of American firms have always shaped how 



people learn to work. In the decades following the 
Second World War, most major firms developed sub- 
stantial training establishments and opportunities, dis- 
tributed principally as a prelude to advancement within 
the firm. Many smaller, particularly high-tech, compa- 
nies came to provide a variety of "off-site" opportuni- 
ties for their most skilled employees to learn new 
techniques and applications. Training in these settings 
became a hallmark of personal success — a signal that 
the employee was truly valued by the firm. For the 
most part, these opportunities were distributed to the 
firms' most educated employees — a clear case of fur- 
ther advantage for the most advantaged. 

Over the years a second and separate training estab- 
lishment has evolved through the action of public poli- 
cy. Federal legislation has created an alphabet soup of 
government initiatives — MDTA, CETA, JTPA, and 
TAA, to name the best known. Like training efforts in 
the private sector, these programs have often lacked 
effective leadership, becoming fragmented or redun- 
dant and frequently falling victim to a shift in political 
winds. From this perspective, the contradictory find- 
ings of the national surveys reporting the incidence of 
training in general and of the CPS and SIPP surveys in 
particular make perfect sense: one inadvertently taps 
into the common wisdom about private-sector training 
and its association with education and personal ad- 
vancement, while the other, in an equally unplanned 
way, triggers reactions to publicly sponsored training. 
Until now, assumptions about the proper relationship of 
analytic data to public policy have rested on the pre- 
sumption that these national data sets measure what 
they were commissioned to measure. 
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Figure 14 

NLS-Y, Any Private Training 1979-1988: All Races • Results of Logit Analysis 
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Figure 15 

NLS-Y, Any Govt Training 1979-1988: All Races • Results of Logit Analysis 
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Conclusions and Recommendations 



We began our explorations assuming that there was a 
less complex set of answers explaining why the national 
surveys produced such contradictory results. We ex- 
pected to separate the wheat from the staff — assumed 
the linguistic cross-walk along with the logistic analy- 
sis would identify the surveys which did the best job. 
What we sought was a reliable set of estimates telling 
us who got trained by whom. 

Our conclusion is that such estimates are not possi- 
ble: that in the technical sense, the surveys do not 
measure fully what they purport to measure. Taken 
collectively, however, the surveys provide an invalu- 
able insight into the nature of work-related education 
and training in the United States: how it is perceived, 
how it is often considered to be contradictory in nature, 
and how important its association with credentialed 
education actually is. . 

These findings come at an important time. The 
growing interest in both the educational quality of the 
workforce and the shifting relationship among jobs, 
skills, and training is creating n demands for both 
public and private initiatives. Fueled by economic 
necessity as well as electoral politics, the emerging 
national debate over the efficacy of the nation's invest- 
ments in work-related education and training on the 
one hand and the need to make the school- to- work tran- 
sition more purposeful and rational on the other is cre- 
ating increased demand for data to measure the current 



scale and scope of how and when the nation invests in 
the educational quality of its workforce. 

Our most specific recommendations — just three in - 
number, each focusing on how to construct more reli- 
able survey instruments — respond to this need for bet- 
ter data. 

1. Questions focusing on the incidence of work- 
related education and training need to be consis- 
tently "bundled" with questions dealing explicitly 
with work, with the kinds of skills the respondent 
needs to know to best perform his or her current 
job, and with the respondent's sense of the role 
training plays within his or her workplace. 

2. Where possible, the surveys themselves ought to 
involve matched samples of employing establish- 
ments (or enterprises) and their employees. For 
the most part, surveys focusing on work-related 
education and training have queried one or the 
other of these populations but seldom both. There 
are important exceptions. A dataset recently 
released by the Survey Research Laboratory at the 
University of Illinois as part of its "National 
Organizations Study" first sampled individuals 
and then queried their employers about the types 
of training they provided, representing an impor- 
tant first step. Another is the World-Bank's 
"Enterprise Training Strategies and Productivity: 
A Cross-National Study," a multinational survey 
of firms and their employees; the American 
survey for this effort is being conducted by EQW. 

3. More explicit questions ought to be developed 
that focus on training credentials and certifi- 
cates — when are they issued, by whom, and with 
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what value. Such questions ought to be asked just 
after questions about educational attainment and 
degrees. 



Our larger set of conclusions and tentative recom- 
mendations — based on our analysis of the national 
surveys as well as the results from related EQW 
projects — involves closing the gap between the two 
kinds of training and related education that the nation- 
al surveys tap. Closing this gap likely requires a con- 
tinuum of job-connected education and training options 
available not only to unskilled and highly-skilled work- 
ers, but also to the broad middle-segment of the na- 
tion's labor force. Even as the decline of traditional 
manufacturing jobs has lessened economic opportuni- 
ties for those who have traditionally sought blue-collar 
careers, the development of new technical occupations 
is offering middle-class security for thousands more. 
Demand for skilled, white-coat technicians — computer 
programmers, medical technicians, paralegals, engi- 
neering technicians, to name just a few — continues to 
grow. 1 * 

The importance of these technicians, however, re- 
mains largely unrecognized. The nature of their occu- 
pations is defined, for now, by a hierarchy that 
excludes them from the ranks of professional staff. As 
a result, such. workers lack clearly defined career 
paths, status, and, most importantly, credentials. 

Indeed credentials may be the key to training reform 
not just for technical crafts but for jobs and workers of 
all types. Growing interest in performance standards 
for secondary and higher education could lead the way 
for a similar reordering of training requirements. Just 
as a high school or college diploma should warrant the 
acquisition of certain predefined skills, comparable 
credentials should vouch for measurable competencies 



gained through accredited private or public training 
programs. 

The creation of portable, standardized Credentials 
would require comprehensive revision of the nation's 
occupational categories and detailed specification of 
skills for each occupation. Considerable time, as well 
as money and imagination, would be necessary to de- 
velop such a system. The likely advantages, however, 
would almost certainly justify the effort. Tying training 
more closely to skills could give new meaning to equal 
opportunity, erasing the stigma attached to public 
training and furnishing powerful incentives for lifelong 
learning. 

Solving the riddle of training statistics will not auto- 
matically solve the problems of work-related education 
and training in the United States. In the unraveling of 
survey discrepancies, however, important clues have 
come to light, illuminating the major challenges of 
training reform. 

In practical terms, closing the gap between public 
and private training could be accomplished through a 
system of national standards and transportable creden- 
tials. Such a system would facilitate skills acquisition 
and occupational mobility not only for individuals at 
either end of the socio-economic spectrum, but also for 
technicians and other front-line workers in the often- 
neglected middle segment of the nation's labor force. It 
is an imposing, but not impossible, challenge. As 
built-in barriers and stereotypes begin to crumble, 
American firms and the workers they employ can rea- 
sonably look forward to a time when effective, job- 
connected skills training — distributed nationwide 
through a network of locally-based, private and public 
suppliers — proves itself to be a vision grounded in 
reality. 
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Endnotes 



1 For a comprehensive sense of the components of this study, 

please read the following publications in conjunction with this 
paper: A Crosswalk of Data Sets Focusing on Worker Trainings 
National Center on the Educational Quality of the Workforce, 
Philadelphia, PA (1992); Statistical Companions to the Cross- 
walk of National Data Sets Focusing on Worker Training, 
Volumes 1 and 2, National Center on the Educational Quality 
of the Workforce, Philadelphia, PA (1993); and "Measuring 
Public Policy: Muddled Mazes and Old Dilemmas" by Robert 
Zenisky, Distinguished Lectures in the Social Sciences, 
Northern Illinois University (1992). 

2 Overview of CPS Sampling Design and Methodology, available 

from the Bureau of the Census. 

3 "How Workers Cet Their Training.*' U.S. Department of Labor, 

Bureau of Labor Statistics. Bulletin 2226, February 1985; 
"How Workers Cet Their Training: A 1991 Update." U.S. 
Department of Labor, Bureau of Labor Statistics. Bulletin 
2407. August 1992. 

4 National Longitudinal Survey of Youth. Washington, DC: U.S. 
Department of Labor, Bureau of Labor Statistics. 

*For a detailed description of all five cohort studies, see the MS 
Handbook 1991, Center for Human Resource Research, The 
Ohio State University. Bureau of Labor Statistics, U.S. Depart- 
ment of Labor, Contract #J-9-J-7-0090. 

6 U,S. Department of Commerce. 1988. Survey of Income and 
Program Participation. Washington, DC: Bureau of the Cen- 
sus. Form S1PP-5020 (6-1-88). 

7 National Center for Educational Statistics. 1987. The National 
longitudinal Study of the High School Class of 1972 (NLS- 
72): Fifth Follow-Up (1986). Washington, DC: Department 
of Education, Office of Educational Research and Improve- 
ment. 

H National Center for Educational Statistics. 1987. High School 
and Beyond 1980 Sophomore Cohort Third Follow-Up. Wash- 
ington, DC: U.S. Department of Education, Office of Educa- 
tional Research and Improvement. 

9 There are some notable exceptions to this tendency to concen- 
trate only oh a single data source. See the work by Lee Lillard 
and Hong Tan, Private Sector Training: Who Gets It and What 
Are Its Effects'? (1986). 

10 Individual income data in the CPS survey are only available for 
' one-quarter of the sample. 



11 Statistical Companions to the Crosswalk of National Data Sets 

Focusing on Worker Training (National Center on the Educa- 
tional Quality of the Workforce 1993) can be obtained by 
contacting the Center. # . 

12 We obtained a description of the logit models from Hanuschek 

and Jackson's Statistical Methods for Social Scientists (1977) 
and the SAS Institute Inc. s SAS/STAT User's Guide (1989). 

M The authors owe a special debt to Joel Levine of Dartmouth 
College for his help in developing the logistical architecture. 
He was simultaneously critic and colleague, equally con- 
cerned with "stretching the envelope" and "getting it right." 

u 0ur knowledge of technical work is drawn principally from the 
work of EQW researcher Stephen Barley of Cornell University. 
Fortune Magazine recently featured Dr. Barley's work on 
technical occupations in their May 17, 1993, cover story, 
"How Will We Work in the Year 2000?" by Walter Kiechel. 
His papers on the changing technical workforce have been 
published by EQW: "The New Crafts: The Rise of the Tech- 
nical Labor Force and Its Implication for the Organization of 
Work" (1992); "Practice Makes Perfect: Emergency Medical 
Technicians and the Social Negotiation of a Skilled Occupa- 
tional Identity" (1992); "In the Backrooms of Science: Notes 
on the Work of Technicians in Science Labs" (1993); and 
"What Do Technicians Do?" (1993). 
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